{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "# How to Use Ollama at Google Colab#  \n",
        " Using Ollama with Google Colab can be tricky! Sometimes our local systems lack the memory or CPU power needed for open-source LLMs. Google Colab offers free GPUs.\n",
        "\n",
        " Here's how to set it up:\n",
        "\n",
        "<br>\n",
        "\n",
        "1️⃣ `!pip install colab-xterm`\n",
        "\n",
        "\n",
        "2️⃣ `%load_ext colabxterm` This is an IPython command (indicated by the % symbol).It loads the colabxterm extension into the current Jupyter notebook environment. Once loaded, this extension allows you to use an interactive terminal within your Colab notebook.\n",
        "\n",
        "<br>\n",
        "\n",
        "(The purpose of both  commands is to set up a terminal environment within the Colab notebook. This can be useful for running command-line tools or interacting with the system in ways that aren't easily done through the standard Colab interface.)\n",
        "\n",
        "\n",
        "3️⃣ `!curl -fsSL https://ollama.com/install.sh | sh`\n",
        "\n",
        "\n",
        "4️⃣ `%xterm` (This opens a terminal in Colab. Run: ollama serve &)\n",
        "\n",
        "\n",
        "5️⃣ `!ollama pull llama3.1 > /dev/null 2>&1`\n",
        "\n",
        "<br>\n",
        "\n",
        "With these commands, you can access Ollama-based LLMs/models on Google Colab."
      ],
      "metadata": {
        "id": "jfDt05fp1eqP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install colab-xterm #https://pypi.org/project/colab-xterm/\n",
        "%load_ext colabxterm"
      ],
      "metadata": {
        "id": "1tgc1KnW1nyH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Ollama Installation\n",
        "\n",
        "This script installs Ollama on your system."
      ],
      "metadata": {
        "id": "9hFuGXWq2W6I"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!curl -fsSL https://ollama.com/install.sh | sh"
      ],
      "metadata": {
        "id": "eKR2hVeoKP9a"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Run `%xterm` Command in Google Colab\n",
        "\n",
        "* This command is used to open an interactive terminal session within a Google Colab notebook.\n",
        "\n",
        "* This allows you to execute command-line tools and interact with the system in ways that aren't easily done through the standard Colab interface.\n",
        "\n",
        "* Use the %xterm command to open the terminal: `%xterm`\n",
        "\n",
        "* Once the terminal is open, you can run any command as if you were using a local terminal. Run following command to start ollama inside colab\n",
        "\n",
        "  `ollama serve &`"
      ],
      "metadata": {
        "id": "lyrcXqJ52_bq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%xterm"
      ],
      "metadata": {
        "id": "y8uMIKoUiQ7s"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Pull Meta's Llama3.1 LLM\n",
        "\n"
      ],
      "metadata": {
        "id": "gObowt7K3S0h"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!ollama pull llama3.1 > /dev/null 2>&1"
      ],
      "metadata": {
        "id": "-gonJ3tiiY7s"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Install Langchain and langchain-ollama libraries"
      ],
      "metadata": {
        "id": "0ifi-FiC3qf7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install -q -U langchain langchain_community langchain-ollama"
      ],
      "metadata": {
        "id": "O5RXAVT9_XW0"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Use Llam3.1 model and langchain to get the LLM response"
      ],
      "metadata": {
        "id": "hhxktS2n3xAS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain_ollama import ChatOllama\n",
        "from langchain.prompts import PromptTemplate\n",
        "from langchain_core.output_parsers import StrOutputParser\n",
        "\n",
        "prompt = PromptTemplate(\n",
        "    template=\"\"\"You are an assistant for question-answering tasks.\n",
        "\n",
        "    If you don't know the answer, just say that you don't know.\n",
        "\n",
        "    Use three sentences maximum and keep the answer concise:\n",
        "    Question: {question}\n",
        "    Answer:\n",
        "    \"\"\",\n",
        "    input_variables=[\"question\"],\n",
        ")\n",
        "\n",
        "llm = ChatOllama(\n",
        "    model=\"llama3.1\",\n",
        "    temperature=0,\n",
        ")\n",
        "\n",
        "chain = prompt | llm | StrOutputParser()\n",
        "response = chain.invoke({\"question\": \"Why should we learn GenAI?\"})\n",
        "print(response)"
      ],
      "metadata": {
        "id": "5cpGA_ayKo4v"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Display a formatted Markdown string in colab or jupyter notebook."
      ],
      "metadata": {
        "id": "viwfWc4f4Gm2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from IPython.display import display, Markdown\n",
        "\n",
        "print(display(Markdown(response)))"
      ],
      "metadata": {
        "id": "mQH05AUzS3WG"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}